<!DOCTYPE html>
<html id="docs" lang="en" class="">
	<head>
	<meta charset="utf-8">
<title>Validation - End-to-end Testing - Kubernetes</title>
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="shortcut icon" type="image/png" href="../../../../images/favicon.png">
<link rel="stylesheet" type="text/css" href="../../../../css/base_fonts.css">
<link rel="stylesheet" type="text/css" href="../../../../css/styles.css">
<link rel="stylesheet" type="text/css" href="https://code.jquery.com/ui/1.12.1/themes/smoothness/jquery-ui.css">
<link rel="stylesheet" type="text/css" href="https://cdnjs.cloudflare.com/ajax/libs/sweetalert/1.1.3/sweetalert.min.css">
<link rel="stylesheet" type="text/css" href="../../../../css/callouts.css">
<link rel="stylesheet" type="text/css" href="../../../../css/custom-jekyll/tags.css">




<meta name="description" content="Validation - End-to-end Testing" />
<meta property="og:description" content="Validation - End-to-end Testing" />

<meta property="og:url" content="https://kubernetes.io/docs/getting-started-guides/ubuntu/validation/" />
<meta property="og:title" content="Validation - End-to-end Testing - Kubernetes" />

<script
src="https://code.jquery.com/jquery-3.2.1.min.js"
integrity="sha256-hwg4gsxgFZhOsEEamdOYGBf13FyQuiTwlAQgxVSNgt4="
crossorigin="anonymous"></script>
<script
src="https://code.jquery.com/ui/1.12.1/jquery-ui.min.js"
integrity="sha256-VazP97ZCwtekAsvgPBSUwPFKdrwD3unUfSGVYrahUqU="
crossorigin="anonymous"></script>
<script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/sweetalert/1.1.3/sweetalert.min.js"></script>
<script src="../../../../js/script.js"></script>
<script src="../../../../js/custom-jekyll/tags.js"></script>


	</head>
	<body>
		<div id="cellophane" onclick="kub.toggleMenu()"></div>

<header>
    <a href="../../../../index.html" class="logo"></a>

    <div class="nav-buttons" data-auto-burger="primary">
        <ul class="global-nav">
            
            
            <li><a href="../../../home.1">Documentation</a></li>
            
            <li><a href="../../../../blog/index.html">Blog</a></li>
            
            <li><a href="../../../../partners/index.html">Partners</a></li>
            
            <li><a href="../../../../community/index.html">Community</a></li>
            
            <li><a href="../../../../case-studies/index.html">Case Studies</a></li>
            
            
             <li>
                <a href="../validation.1#">
                    English <span class="ui-icon ui-icon-carat-1-s"></span>
                </a>
                <ul>
                
                    <li><a href="../../../../zh/index.html">中文 Chinese</a></li>
                
                    <li><a href="../../../../ko/index.html">한국어 Korean</a></li>
                
                </ul>
            </li>
         
            <li>
                <a href="../validation.1#">
                    v1.11 <span class="ui-icon ui-icon-carat-1-s"></span>
                </a>
                <ul>
                
                    <li><a href="https://kubernetes.io">v1.12</a></li>
                
                    <li><a href="../../../../index.html">v1.11</a></li>
                
                    <li><a href="https://v1-10.docs.kubernetes.io">v1.10</a></li>
                
                    <li><a href="https://v1-9.docs.kubernetes.io">v1.9</a></li>
                
                </ul>
            </li>
        </ul>
        
        <a href="../../../tutorials/kubernetes-basics/index.html" class="button" id="tryKubernetes" data-auto-burger-exclude>Try Kubernetes</a>
        <button id="hamburger" onclick="kub.toggleMenu()" data-auto-burger-exclude><div></div></button>
    </div>

    <nav id="mainNav">
        <main data-auto-burger="primary">
        <div class="nav-box">
            <h3><a href="../../../tutorials/stateless-application/hello-minikube/index.html">Get Started</a></h3>
            <p>Ready to get your hands dirty? Build a simple Kubernetes cluster that runs "Hello World" for Node.js.</p>
        </div>
        <div class="nav-box">
            <h3><a href="../../../home.1">Documentation</a></h3>
            <p>Learn how to use Kubernetes with the use of walkthroughs, samples, and reference documentation. You can even <a href="../../../../editdocs/index.html" data-auto-burger-exclude>help contribute to the docs</a>!</p>
        </div>
        <div class="nav-box">
            <h3><a href="../../../../community/index.html">Community</a></h3>
            <p>If you need help, you can connect with other Kubernetes users and the Kubernetes authors, attend community events, and watch video presentations from around the web.</p>
        </div>
        <div class="nav-box">
            <h3><a href="../../../../blog/index.html">Blog</a></h3>
            <p>Read the latest news for Kubernetes and the containers space in general, and get technical how-tos hot off the presses.</p>
        </div>
        </main>
        <main data-auto-burger="primary">
        <div class="left">
            <h5 class="github-invite">Interested in hacking on the core Kubernetes code base?</h5>
            <a href="https://github.com/kubernetes/kubernetes" class="button" data-auto-burger-exclude>View On Github</a>
        </div>

        <div class="right">
            <h5 class="github-invite">Explore the community</h5>
            <div class="social">
                <a href="https://twitter.com/kubernetesio" class="twitter"><span>Twitter</span></a>
                <a href="https://github.com/kubernetes/kubernetes" class="github"><span>Github</span></a>
                <a href="http://slack.k8s.io/" class="slack"><span>Slack</span></a>
                <a href="http://stackoverflow.com/questions/tagged/kubernetes" class="stack-overflow"><span>Stack Overflow</span></a>
                <a href="https://discuss.kubernetes.io" class="mailing-list"><span>Forum</span></a>
                <a href="https://calendar.google.com/calendar/embed?src=nt2tcnbtbied3l6gi2h29slvc0%40group.calendar.google.com" class="calendar"><span>Events Calendar</span></a>
            </div>
        </div>
        <div class="clear" style="clear: both"></div>
        </main>
    </nav>
</header>

		
		
		<section id="hero" class="light-text no-sub">
			
















<div id="vendorStrip" class="light-text">
	<ul>
		
		
		<li><a href="../../../home.1">DOCUMENTATION</a></li>
		
		
		<li><a href="../../../setup/index.html">SETUP</a></li>
		
		
		<li><a href="../../../concepts/index.html">CONCEPTS</a></li>
		
		
		<li><a href="../../../tasks/index.html">TASKS</a></li>
		
		
		<li><a href="../../../tutorials/index.html">TUTORIALS</a></li>
		
		
		<li><a href="../../../reference.1">REFERENCE</a></li>
		
	</ul>
	<div id="searchBox">
		<input type="text" id="search" placeholder="Search" onkeydown="if (event.keyCode==13) window.location.replace('/docs/search/?q=' + this.value)" autofocus="autofocus">
	</div>
</div>

		</section>
		
		
<section id="deprecationWarning">
  <main>
    <div class="content deprecation-warning">
      <h3>
        Documentation for Kubernetes v1.11 is no longer actively maintained. The version you are currently viewing is a static snapshot.
        For up-to-date documentation, see the <a href="https://kubernetes.io/docs/home/">latest</a> version.
      </h3>
    </div>
  </main>
</section>


		<section id="encyclopedia">
			
<div id="docsToc">
     <div class="pi-accordion">
    	
        
        
        
        
        
         
             
                 
             
         
             
                 
             
         
             
                 
             
         
             
                 
                          
                          
                 
             
         
             
         
             
         
             
         
             
         
         
        
        <a class="item" data-title="Independent Solutions" href="../../index.html"></a>

	
	
		
		
	<div class="item" data-title="Bare Metal">
		<div class="container">
		
		
	
	
		
		
<a class="item" data-title="Fedora (Multi Node)" href="../../fedora/flannel_multi_node_cluster/index.html"></a>

		
	
		
		
<a class="item" data-title="Fedora (Single Node)" href="../../fedora/fedora_manual_config.1"></a>

		
	

		</div>
	</div>

		
	
		
		
	<div class="item" data-title="Kubernetes on Ubuntu">
		<div class="container">
		
		
	
	
		
		
<a class="item" data-title="Backups" href="../backups.1"></a>

		
	
		
		
<a class="item" data-title="Decommissioning" href="../decommissioning/index.html"></a>

		
	
		
		
<a class="item" data-title="Glossary and Terminology" href="../glossary/index.html"></a>

		
	
		
		
<a class="item" data-title="Local Kubernetes development with LXD" href="../local/index.html"></a>

		
	
		
		
<a class="item" data-title="Logging" href="../logging.1"></a>

		
	
		
		
<a class="item" data-title="Monitoring" href="../monitoring/index.html"></a>

		
	
		
		
<a class="item" data-title="Networking" href="../networking/index.html"></a>

		
	
		
		
<a class="item" data-title="Operational Considerations" href="../operational-considerations/index.html"></a>

		
	
		
		
<a class="item" data-title="Rancher Integration with Ubuntu Kubernetes" href="../rancher/index.html"></a>

		
	
		
		
<a class="item" data-title="Scaling" href="../scaling/index.html"></a>

		
	
		
		
<a class="item" data-title="Security Considerations" href="../security/index.html"></a>

		
	
		
		
<a class="item" data-title="Setting up Kubernetes with Juju" href="../installation/index.html"></a>

		
	
		
		
<a class="item" data-title="Storage" href="../storage/index.html"></a>

		
	
		
		
<a class="item" data-title="Troubleshooting" href="../troubleshooting/index.html"></a>

		
	
		
		
<a class="item" data-title="Upgrades" href="../upgrades/index.html"></a>

		
	
		
		
<a class="item" data-title="Validation - End-to-end Testing" href="../validation.1"></a>

		
	

		</div>
	</div>

		
	
		
		
<a class="item" data-title="Deprecated Alternatives" href="../../alternatives/index.html"></a>

		
	






     </div> 
    <button class="push-menu-close-button" onclick="kub.toggleToc()"></button>
</div> 

			<div id="docsContent">
				
<p><a href="../../../editdocs#docs/getting-started-guides/ubuntu/validation.md" id="editPageButton">Edit This Page</a></p>

<h1>Validation - End-to-end Testing</h1>




<p>This page will outline how to ensure that a Juju-deployed Kubernetes
cluster has stood up correctly and is ready to accept workloads.</p>











<ul id="markdown-toc">










<li><a href="../validation.1#before-you-begin">Before you begin</a></li>












<li><a href="../validation.1#end-to-end-testing">End-to-end testing</a></li>




<li><a href="../validation.1#upgrading-the-e2e-tests">Upgrading the e2e tests</a></li>



























</ul>



<h2 id="before-you-begin">Before you begin</h2>
<p>This page assumes you have a working Juju deployed cluster.</p>




<h2 id="end-to-end-testing">End-to-end testing</h2>

<p>End-to-end (e2e) tests for Kubernetes provide a mechanism to test end-to-end
behavior of the system, and is the last signal to ensure end user operations
match developer specifications. Although unit and integration tests provide a
good signal, in a distributed system like Kubernetes it is not uncommon that a
minor change may pass all unit and integration tests, but cause unforeseen
changes at the system level.</p>

<p>The primary objectives of the e2e tests are to ensure a consistent and reliable
behavior of the kubernetes code base, and to catch hard-to-test bugs before
users do, when unit and integration tests are insufficient.</p>

<p>End-to-end tests will pass on a properly running CDK cluster outside of bugs in the tests.</p>

<h3 id="deploy-kubernetes-e2e-charm">Deploy kubernetes-e2e charm</h3>

<p>To deploy the end-to-end test suite, you need to relate the <code>kubernetes-e2e</code> charm
to your existing kubernetes-master nodes and easyrsa:</p>

<pre><code>juju deploy cs:~containers/kubernetes-e2e
juju add-relation kubernetes-e2e easyrsa
juju add-relation kubernetes-e2e:kubernetes-master kubernetes-master:kube-api-endpoint
juju add-relation kubernetes-e2e:kube-control kubernetes-master:kube-control
</code></pre>

<p>Once the relations have settled, you can do <code>juju status</code> until the workload status results in
 <code>Ready to test.</code> - you may then kick off an end to end validation test.</p>

<h3 id="running-the-e2e-test">Running the e2e test</h3>

<p>The e2e test is encapsulated as an action to ensure consistent runs of the
end to end test. The defaults are sensible for most deployments.</p>

<pre><code>juju run-action kubernetes-e2e/0 test
</code></pre>

<h3 id="tuning-the-e2e-test">Tuning the e2e test</h3>

<p>The e2e test is configurable. By default it will focus on or skip the declared
conformance tests in a cloud agnostic way. Default behaviors are configurable.
This allows the operator to test only a subset of the conformance tests, or to
test more behaviors not enabled by default. You can see all tunable options on
the charm by inspecting the schema output of the actions:</p>

<pre><code>juju actions kubernetes-e2e --format=yaml --schema
</code></pre>

<p>Output:</p>

<pre><code>test:
  description: Run end-to-end validation test suite
  properties:
    focus:
      default: \[Conformance\]
      description: Regex focus for executing the test
      type: string
    skip:
      default: \[Flaky\]
      description: Regex of tests to skip
      type: string
    timeout:
      default: 30000
      description: Timeout in nanoseconds
      type: integer
  title: test
  type: object
</code></pre>

<p>As an example, you can run a more limited set of tests for rapid validation of
a deployed cluster. The following example will skip the <code>Flaky</code>, <code>Slow</code>, and
<code>Feature</code> labeled tests:</p>

<pre><code>juju run-action kubernetes-e2e/0 test skip='\[(Flaky|Slow|Feature:.*)\]'
</code></pre>

<blockquote class="note">
  <div><strong>Note:</strong> the escaping of the regex due to how bash handles brackets.</div>
</blockquote>

<p>To see the different types of tests the Kubernetes end-to-end charm has access
to, we encourage you to see the <a href="https://git.k8s.io/community/contributors/devel/e2e-tests.md#kinds-of-tests" target="_blank">upstream documentation on the different types
of tests</a>,
and to thoroughly understand what subsets of the tests you are running.</p>

<h3 id="more-information-on-end-to-end-testing">More information on end-to-end testing</h3>

<p>Along with the above descriptions, end-to-end testing is a much larger subject
than this readme can encapsulate. There is far more information in the
<a href="https://git.k8s.io/community/contributors/devel/e2e-tests.md" target="_blank">end-to-end testing guide</a>.</p>

<h3 id="evaluating-end-to-end-results">Evaluating end-to-end results</h3>

<p>It is not enough to just simply run the test. Result output is stored in two
places. The raw output of the e2e run is available in the <code>juju show-action-output</code>
command, as well as a flat file on disk on the <code>kubernetes-e2e</code> unit that
executed the test.</p>

<blockquote class="note">
  <div><strong>Note:</strong> The results will only be available once the action has
completed the test run. End-to-end testing can be quite time consuming, often
taking more than 1 hour, depending on configuration.</div>
</blockquote>

<h5 id="accessing-the-results-in-a-flat-file">Accessing the results in a flat file</h5>

<p>Here&rsquo;s how to copy the output out as a file:</p>

<pre><code>juju run-action kubernetes-e2e/0 test
</code></pre>

<p>Output:</p>

<pre><code>Action queued with id: 4ceed33a-d96d-465a-8f31-20d63442e51b
</code></pre>

<p>Copy output to your local machine:</p>

<pre><code>juju scp kubernetes-e2e/0:4ceed33a-d96d-465a-8f31-20d63442e51b.log .
</code></pre>

<h5 id="action-result-output">Action result output</h5>

<p>Or you can just show the output inline:</p>

<pre><code>juju run-action kubernetes-e2e/0 test
</code></pre>

<p>Output:</p>

<pre><code>Action queued with id: 4ceed33a-d96d-465a-8f31-20d63442e51b
</code></pre>

<p>Show the results in your terminal:</p>

<pre><code>juju show-action-output 4ceed33a-d96d-465a-8f31-20d63442e51b
</code></pre>

<h3 id="known-issues">Known issues</h3>

<p>The e2e test suite assumes egress network access. It will pull container
images from <code>gcr.io</code>. You will need to have this registry unblocked in your
firewall to successfully run e2e test results. Or you may use the exposed
proxy settings <a href="https://github.com/juju-solutions/bundle-canonical-kubernetes#proxy-configuration" target="_blank">properly configured</a>
on the kubernetes-worker units.</p>

<h2 id="upgrading-the-e2e-tests">Upgrading the e2e tests</h2>

<p>The e2e tests are always expanding; you can see if there&rsquo;s an upgrade
available by running <code>juju status kubernetes-e2e</code>.</p>

<p>When an upgrade is available, upgrade your deployment:</p>

<pre><code>juju upgrade-charm kubernetes-e2e
</code></pre>



















				<div class="issue-button-container">
					<p><a href="../validation.1"><img src="https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/getting-started-guides/ubuntu/validation.md?pixel" alt="Analytics" /></a></p>
					
					
					<script type="text/javascript">
					PDRTJS_settings_8345992 = {
					"id" : "8345992",
					"unique_id" : "\/docs\/getting-started-guides\/ubuntu\/validation\/",
					"title" : "Validation - End-to-end Testing",
					"permalink" : "https:\/\/kubernetes.io\/docs\/getting-started-guides\/ubuntu\/validation\/"
					};
					(function(d,c,j){if(!document.getElementById(j)){var pd=d.createElement(c),s;pd.id=j;pd.src=('https:'==document.location.protocol)?'https://polldaddy.com/js/rating/rating.js':'http://i0.poll.fm/js/rating/rating.js';s=document.getElementsByTagName(c)[0];s.parentNode.insertBefore(pd,s);}}(document,'script','pd-rating-js'));
					</script>
					<a href="../validation.1" onclick="window.open('https://github.com/kubernetes/website/issues/new?title=Issue%20with%20' +
					'k8s.io'+window.location.pathname)" class="button issue">Create an Issue</a>
					
					
					
					<a href="../../../editdocs#docs/getting-started-guides/ubuntu/validation.md" class="button issue">Edit this Page</a>
					
				</div>
			</div>
		</section>
		<footer>
    <main class="light-text">
        <nav>
            
            
            
            <a href="../../../home.1">Documentation</a>
            
            <a href="../../../../blog/index.html">Blog</a>
            
            <a href="../../../../partners/index.html">Partners</a>
            
            <a href="../../../../community/index.html">Community</a>
            
            <a href="../../../../case-studies/index.html">Case Studies</a>
            
        </nav>
        <div class="social">
            <div>
                <a href="https://twitter.com/kubernetesio" class="twitter"><span>twitter</span></a>
                <a href="https://github.com/kubernetes/kubernetes" class="github"><span>Github</span></a>
                <a href="http://slack.k8s.io/" class="slack"><span>Slack</span></a>
            </div>
            <div>
                <a href="http://stackoverflow.com/questions/tagged/kubernetes" class="stack-overflow"><span>Stack Overflow</span></a>
                <a href="https://discuss.kubernetes.io" class="mailing-list"><span>Forum</span></a>
                <a href="https://calendar.google.com/calendar/embed?src=nt2tcnbtbied3l6gi2h29slvc0%40group.calendar.google.com" class="calendar"><span>Events Calendar</span></a>
            </div>
            <div>
                <a href="../../index.html" class="button">Get Kubernetes</a>
                <a href="https://git.k8s.io/community/contributors/guide" class="button">Contribute</a>
            </div>
        </div>
        <div id="miceType" class="center">
            &copy; 2018 The Kubernetes Authors | Documentation Distributed under <a href="https://git.k8s.io/website/LICENSE" class="light-text">CC BY 4.0</a>
        </div>
        <div id="miceType" class="center">
            Copyright &copy; 2018 The Linux Foundation&reg;. All rights reserved. The Linux Foundation has registered trademarks and uses trademarks. For a list of trademarks of The Linux Foundation, please see our <a href="https://www.linuxfoundation.org/trademark-usage" class="light-text">Trademark Usage page</a>
        </div>
    </main>
</footer>

		<button class="flyout-button" onclick="kub.toggleToc()"></button>

<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
    (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-36037335-10', 'auto');
ga('send', 'pageview');


(function () {
    window.addEventListener('DOMContentLoaded', init)

        
        function init() {
            window.removeEventListener('DOMContentLoaded', init)
                hideNav()
        }

    function hideNav(toc){
        if (!toc) toc = document.querySelector('#docsToc')
        if (!toc) return
            var container = toc.querySelector('.container')

                
                if (container) {
                    if (container.childElementCount === 0 || toc.querySelectorAll('a.item').length === 1) {
                        toc.style.display = 'none'
                            document.getElementById('docsContent').style.width = '100%'
                    }
                } else {
                    requestAnimationFrame(function () {
                        hideNav(toc)
                    })
                }
    }
})();
</script>



	</body>
</html>